CausaLM: Causal Model Explanation Through Counterfactual Language Models
نویسندگان
چکیده
Understanding predictions made by deep neural networks is notoriously difficult, but also crucial to their dissemination. As all machine learning based methods, they are as good training data, and can capture unwanted biases. While there tools that help understand whether such biases exist, do not distinguish between correlation causation, might be ill-suited for text-based models reasoning about high level language concepts. A key problem of estimating the causal effect a concept interest on given model this estimation requires generation counterfactual examples, which challenging with existing technology. To bridge gap, we propose CausaLM, framework producing explanations using representation models. Our approach fine-tuning contextualized embedding auxiliary adversarial tasks derived from graph problem. Concretely, show carefully choosing pre-training tasks, BERT effectively learn interest, used estimate its true performance. byproduct our method unaffected tested concept, useful in mitigating bias ingrained data.
منابع مشابه
Running head: CAUSAL AND COUNTERFACTUAL EXPLANATION Mental Simulation and the Nexus of Causal and Counterfactual Explanation
متن کامل
From Causal Models To Counterfactual Structures
Galles and Pearl [1998] claimed that “for recursive models, the causal model framework does not add any restrictions to counterfactuals, beyond those imposed by Lewis’s [possible-worlds] framework.” This claim is examined carefully, with the goal of clarifying the exact relationship between causal models and Lewis’s framework. Recursive models are shown to correspond precisely to a subclass of ...
متن کاملCausal Explanation from Mathematical Models
Decision support systems may require the use of existing complex mathematical models. It is desirable to reduce the equations of such a model to an explanatory causal form to support decision making, for example in making decisions about the greenhouse effect. This paper presents an approach to constructing causal explanation from existing complex mathematical models. The method combines inform...
متن کاملGenerating Simplest Models for Causal Explanation
Central to most scientific and engineering tasks design, diagnosis, analogy and design rationale capture are descriptions of a device that capture its salient features with respect to how it works . These descriptions are used as the central focus of many engineering reasoning techniques . Yet little has been said about what constitutes a best, or even adequate, description . Likewise, related ...
متن کاملCounterfactual Language Model Adaptation for Suggesting Phrases
Mobile devices use language models to suggest words and phrases for use in text entry. Traditional language models are based on contextual word frequency in a static corpus of text. However, certain types of phrases, when offered to writers as suggestions, may be systematically chosen more often than their frequency would predict. In this paper, we propose the task of generating suggestions tha...
متن کاملذخیره در منابع من
با ذخیره ی این منبع در منابع من، دسترسی به آن را برای استفاده های بعدی آسان تر کنید
ژورنال
عنوان ژورنال: Computational Linguistics
سال: 2021
ISSN: ['1530-9312', '0891-2017']
DOI: https://doi.org/10.1162/coli_a_00404